Information processing apparatus and information processing method

ABSTRACT

An information processing apparatus includes an additional information conversion means for converting a text of related information into additional information and outputting this additional information, and for outputting an amount of additional information for each point of interest, an embedding region allocating means for outputting region management information including the amount of additional information, for allocating an embedding region for the additional information in an original digital input image on the basis of image size information and the amount of additional information, and for outputting information on the embedding region as region information, and an information embedding means for embedding the region management information in a predetermined region in the original digital input image, and also embedding the additional information in the original digital input image on the basis of the region information to generate and output a watermarked image.

FIELD OF THE INVENTION

The present invention relates to an information processing apparatus forand an information processing method of embedding related information ona point of interest in a digital image by means of an electronicwatermarking method, or detecting related information embedded in adigital image.

BACKGROUND OF THE INVENTION

A prior art information processing apparatus, as disclosed in, forexample, patent reference 1, allows the user to specify a region ofinterest first, acquires information on the position of the region ofinterest, and forms additional information, as well as informationattached to the position information (i.e., utilization information).The prior art information processing apparatus then embeds theadditional information, just as it is, in an embedding region oforiginal image data, which can be the whole of the original image dataor the specified region of interest, by means of an electronicwatermarking method, and manages the image data and additionalinformation without discriminating between them. This patent reference 1also discloses a patchwork method and a basic electronic watermarkingmethod associated with reliability distance. Using the patchwork methodand basic electronic watermarking method, the prior art informationprocessing apparatus carries out highlighting of the boundary of theregion of interest in which the additional information is embedded toproduce a display of the position of the detected region of interest onthe image.

Patent reference 1: JP, 2002-190945, A (see paragraph numbers 0058 to0061, 0065 to 0081, and 0158)

A problem with the prior art information processing apparatusconstructed as mentioned above is that while the user is allowed tospecify a region of interest first, because in a mode in which the priorart information processing apparatus embeds additional information inonly the region of interest, the prior art information processingapparatus does not refer to the amount of the additional information tobe embedded in the region of interest, and therefore must provide amargin for the specified region of interest to make it larger in a stagewhere the amount of the additional information to be embedded in theregion of interest is undecided, the prior art information processingapparatus cannot embed the additional information in the region ofinterest efficiently especially in a case in which the amount of theadditional information to be embedded in the image is restricted.

Another problem is that because the prior art information processingapparatus must specify the region of interest on the image so as to makeit larger, it cannot show only the intended region of interesteffectively.

The present invention is made in order to solve the above-mentionedproblems, and it is therefore an object of the present invention toprovide an information processing apparatus for and an informationprocessing method of referring to the amount of additional informationincluding related information on a point of interest on an imagespecified by the user and allocating an embedding region required forembedding of the additional information appropriately so that theadditional information can be embedded efficiently in a whole imageregion, and only the point of interest, as an intended region ofinterest, and the related information can be shown effectively.

DISCLOSURE OF THE INVENTION

An information processing apparatus in accordance with the presentinvention includes: an image display means for displaying an originaldigital input image, and for outputting information on a size of theimage; a point of interest specifying means for specifying a point ofinterest in the original digital input image currently displayed on abasis of an inputted instruction; a related information output means forinputting related information on the above-mentioned point of interest,and for outputting a text of the related information; an additionalinformation conversion means for converting the text of theabove-mentioned related information to additional information andoutputting this additional information, and for outputting an amount ofthe additional information on the above-mentioned each point ofinterest; an embedding region allocating means for outputting regionmanagement information including the above-mentioned amount ofadditional information, for allocating an embedding region for theabove-mentioned additional information in the above-mentioned originaldigital input image on a basis of the above-mentioned image sizeinformation and the above-mentioned amount of the additionalinformation, and for outputting information on the above-mentionedembedding region as region information; and an embedding informationmeans for embedding the above-mentioned region management information ina predetermined region in the above-mentioned original digital inputimage, and also embedding the above-mentioned additional information inthe above-mentioned original digital input image on a basis of theabove-mentioned region information to generate and output an watermarkedimage.

Therefore, the present embodiment offers an advantage of being able toembed the additional information into the whole image regionefficiently, thereby being able to show only an intended region ofinterest effectively.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing the structure of an informationprocessing apparatus in accordance with embodiment 1 of the presentinvention which is an embedding side;

FIG. 2 is a block diagram showing the structure of an informationprocessing apparatus in accordance with embodiment 1 of the presentinvention which is a detection side;

FIG. 3 is a diagram showing examples of the formats of region managementinformation and additional information in a case in which coordinateinformation on each point of interest is included in the regionmanagement information in the information processing apparatus inaccordance with embodiment 1 of the present invention;

FIG. 4 is a diagram showing examples of the formats of the regionmanagement information and additional information in a case in which thecoordinate information on each point of interest is included in theadditional information in the information processing apparatus inaccordance with embodiment 1 of the present invention;

FIG. 5 is a conceptual diagram for explaining an example of processingcarried out by the information processing apparatus in accordance withembodiment 1 of the present invention which is the embedding side;

FIG. 6 is a block diagram showing the internal structure of aninformation embedding means of the information processing apparatus inaccordance with embodiment 1 of the present invention which is theembedding side;

FIG. 7 is a conceptual diagram for explaining another example of theprocessing carried out by the information processing apparatus inaccordance with embodiment 1 of the present invention which is theembedding side;

FIG. 8 is a block diagram showing the internal structure of theinformation embedding means of the information processing apparatus inaccordance with embodiment 1 of the present invention which is theembedding side;

FIG. 9 is a conceptual diagram for explaining an example of processingcarried out by the information processing apparatus in accordance withembodiment 1 of the present invention which is the detection side;

FIG. 10 is a block diagram showing the internal structure of aninformation detecting means of the information processing apparatus inaccordance with embodiment 1 of the present invention which is thedetection side;

FIG. 11 is a block diagram showing the internal structure of theinformation detecting means of the information processing apparatus inaccordance with embodiment 1 of the present invention which is thedetection side;

FIG. 12 is a block diagram showing the internal structure of theinformation detecting means of the information processing apparatus inaccordance with embodiment 1 of the present invention which is thedetection side;

FIG. 13 is a block diagram showing the structure of an informationprocessing apparatus in accordance with embodiment 2 of the presentinvention which is an embedding side;

FIG. 14 is a block diagram showing the structure of an informationprocessing apparatus in accordance with embodiment 3 of the presentinvention which is an embedding side;

FIG. 15 is a block diagram showing the structure of an informationprocessing apparatus in accordance with embodiment 3 of the presentinvention which is a detection side;

FIG. 16 is a diagram showing examples of the format of additionalinformation in a case in which coordinate information on each point ofinterest is included in region management information in the informationprocessing apparatus in accordance with embodiment 3 of the presentinvention;

FIG. 17 is a diagram showing examples of the format of the additionalinformation in a case in which coordinate information on each point ofinterest is included in the additional information in the informationprocessing apparatus in accordance with embodiment 3 of the presentinvention;

FIG. 18 is a diagram showing examples of the format of the regionmanagement information in a case in which only a text of relatedinformation on each point of interest is included in the additionalinformation in the information processing apparatus in accordance withembodiment 3 of the present invention;

FIG. 19 is a diagram showing an example of the format of the additionalinformation in which a plurality of data fields are successivelyarranged for each point of interest in the information processingapparatus in accordance with embodiment 3 of the present invention;

FIG. 20 is a conceptual diagram for explaining an example of processingperformed by the information processing apparatus in accordance withembodiment 3 of the present invention which is the embedding side;

FIG. 21 is a conceptual diagram for explaining another example of theprocessing performed by the information processing apparatus inaccordance with embodiment 3 of the present invention which is theembedding side;

FIG. 22 is a block diagram showing the structure of informationprocessing apparatus in accordance with embodiment 4 of the presentinvention which is an embedding side;

FIG. 23 is a block diagram showing the structure of a digital camera inaccordance with embodiment 5 in which an information processingapparatus in accordance with the present invention which is an embeddingside is mounted;

FIG. 24 is a block diagram showing the structure of a camera-equippedmobile phone or PC in accordance with embodiment 6 in which aninformation processing apparatus in accordance with the presentinvention which is an embedding side is mounted;

FIG. 25 is a block diagram showing the structure of a camera-equippedmobile phone or PC in accordance with embodiment 6 in which aninformation processing apparatus in accordance with the presentinvention which is a detection side is mounted;

FIG. 26 is a block diagram showing the structure of a printer inaccordance with embodiment 7 in which an information processingapparatus in accordance with the present invention which is a detectionside is mounted.

PREFERRED EMBODIMENTS OF THE INVENTION

Hereafter, in order to explain this invention in greater detail, thepreferred embodiments of the present invention will be described withreference to the accompanying drawings.

Embodiment 1

FIG. 1 is a block diagram showing the structure of an informationprocessing apparatus 10 in accordance with embodiment 1 of the presentinvention which serves as an embedding side, for embedding regionmanagement information and additional information in an image. Thisinformation processing apparatus 10 is provided with an image displaymeans 11, a point of interest specifying means 12, a related informationinput means 13, a point of interest input switching means 14, anadditional information conversion means 15, an embedding regionallocating means 16, and an information embedding means 17.

Next, the operation of the information processing apparatus 10 which isan embedding side will be explained.

The image display means 11 displays an original digital input image 101inputted thereto, and outputs image size information 102 on the size ofthe original digital input image 101. The point of interest specifyingmeans 12 specifies a desired point of interest on the original digitalinput image 101 currently being displayed by the image display means 11according to the user's input, and outputs coordinate information 103 onthe coordinates of the point of interest. When specifying a markindicating the point of interest on the image if needed, the point ofinterest specifying means 12 also specifies and outputs mark information104 on each point of interest either according to the user's input in acase of specifying the shape, such as a quadrangle or a circle, and sizeof the mark, or automatically in a case of placing, as the mark, apredetermined shape, such as a point.

The related information input means 13 inputs related informationassociated with the specified point of interest from the user, andoutputs a text 105 of the related information. The point of interestinput switching means 14 makes the point of interest specifying means12, related information input means 13, and embedding region allocatingmeans 16 carry their respective processes for each point of interestaccording to the user's specifications.

The additional information conversion means 15 accepts the markinformation 104 on each point of interest delivered thereto from thepoint of interest specifying means 12 if needed, and the text 105 of therelated information from the related information input means 13, andconverts the mark information 104 on each point of interest deliveredthereto if needed, and the text 105 of the related information intoformats of additional information 106 and outputs them. The additionalinformation conversion means 15 also outputs an amount 107 of theadditional information for each point of interest.

As shown by a dotted line indicating the coordinate information 103 oneach point of interest in FIG. 1, the additional information conversionmeans 15 inputs the coordinate information 103 on each point of interestfrom the point of interest specifying means 12, the mark information 104on each point of interest from the point of interest specifying means 12if needed, and the text 105 of the related information from the relatedinformation input means 13, and converts the coordinate information 103on each point of interest, mark information 104 on each point ofinterest if needed, and the text 105 of the related information toformats of additional information 106, and outputs them. The additionalinformation conversion means 15 can output an amount 107 of additionalinformation on each point of interest.

The embedding region allocating means 16 accepts the image sizeinformation 102 from the image display means 11, the coordinateinformation 103 on each point of interest from the point of interestspecifying means 12, and the amount 107 of additional information foreach point of interest from the additional information conversion means15, and allocates an embedding region for the additional information 106on the original digital input image 101 on the basis of the image sizeinformation 102 and the amount 107 of additional information on eachpoint of interest. The embedding region allocating means 16 alsogenerates and outputs region management information 108 including boththe coordinate information 103 on each point of interest and the amount107 of additional information on each point of interest, and outputsregion information 109 indicating the embedding region for theadditional information 106.

In a case in which the coordinate information 103 on each point ofinterest from the point of interest specifying means 12 is outputted tothe additional information conversion means 15, the embedding regionallocating means 16 accepts the image size information 102 from theimage display means 11, and the amount 107 of additional information oneach point of interest from the additional information conversion means15, and allocates an embedding region for the additional information 106on the original digital input image 101 on the basis of the image sizeinformation 102 and the amount 107 of additional information on eachpoint of interest. The embedding region allocating means 16 alsogenerates and outputs region management information 108 including theamount 107 of additional information on each point of interest, andoutputs region information 109 indicating the embedding region for theadditional information 106.

The information embedding means 17 embeds the region managementinformation 108 from the embedding region allocating means 16 in apredetermined region of the original digital input image 101 by means ofan electronic watermarking method, and also embeds the additionalinformation 106 from the additional information conversion means 15 inthe original digital input image 101 on the basis of the regioninformation 109 from the embedding region allocating means 16 by meansof the electronic watermarking method, so as to output a watermarkedimage 110.

In this case, the information embedding means 17 embeds the regionmanagement information 108 from the embedding region allocating means 16in the predetermined region of the original digital input image 101. Asan alternative, the embedding region allocating means 16 can allocate anembedding region for the region management information 108 according toan instruction from the user, and incorporate it into the regioninformation 109 and output this information, and the informationembedding means 17 can also embed the region management information 108in the predetermined region on the basis of the region information 109from the embedding region allocating means 16. In this case, theembedding region for the region management information 108 is notifiedto a detection side, and the information is shared between the embeddingside and the detection side.

The embedding region allocating means 16 can also refer to the amount107 of additional information on each point of interest, and allocate anoptimal region size of embedding region for the additional information106 to be embedded in the original digital input image 101 withoutproviding any redundancy for the region size. As an alternative,redundancy can be intentionally provided for the region size. Forexample, the embedding region for the additional information isallocated so that a fixed quantity or a fixed ratio of redundancy can beprovided for the region size of the embedding region. Alternatively, theembedding region for the additional information is allocated so thatmultiple of unit sizes, such as multiple bytes or words, of redundancycan be provided for the region size of the embedding region.

After a cryptographic key is established between the embedding anddetection sides, the region management information 108 and additionalinformation 106 can be embedded in the original input image by means ofthe electronic watermarking method after being encrypted using the key,and they can be decrypted after being detected. In this case, theembedding region allocating means 16 and additional informationconversion means 15 can carry out encryption of the region managementinformation 108 and additional information 106 as a stage preceding thestage of embedding by means of the electronic watermarking method, andan embedding region allocating means 23 and an additional informationreverse conversion means 24, which are disposed as a stage next to thestage of embedding by means of the electronic watermarking method, of abelow-mentioned information processing apparatus 20, which serves as thedetection side as shown in FIG. 2, can have a function of carrying outdecryption of the region management information 108 and additionalinformation 106.

FIG. 2 is a block diagram showing the structure of an informationprocessing apparatus 20 in accordance with embodiment 1 of the presentinvention which serves as a detection side for detecting the regionmanagement information and additional information. This informationprocessing apparatus 20 which is a detection side is provided with aninformation detection means 21, an image display means 22, the embeddingregion allocating means 23, the additional information reverseconversion means 24, a point of interest display means 25, a relatedinformation display means 26, a display position determining means 27,and a point of interest display switching means 28.

Next, the operation of the information processing apparatus 20 which isa detection side will be explained.

The information detection means 21 detects the region managementinformation 108 which is embedded in the predetermined region, which hasbeen notified thereto from the information processing apparatus 10 whichis the embedding side, by means of the electronic watermarking method,from the inputted watermarked image 110, and outputs a digitized outputimage 121.

If the region management information 108 is not embedded in thepredetermined region, the embedding region allocating means 23 outputsregion information 109 indicating the embedding region for the regionmanagement information 108 on the basis of an instruction from the userwhich has been notified thereto from the embedding side, and theinformation detection means 21 detects the region management information108 embedded in the embedding region on the basis of the regioninformation 109 from the inputted watermarked image 110.

The image display means 22 displays the digitized output image 121outputted from the information detection means 21, and outputs imagesize information 102 on the digitized output image 121.

The embedding region allocating means 23 allocates the embedding regionfor the additional information 106 on the digitized output image 121 onthe basis of both the image size information 102 from the image displaymeans 22 and the amount 107 of additional information on each point ofinterest included in the region management information 108 detected bythe information detection means 21, and outputs region information 109indicating the embedding region for the additional information 106 andalso outputs the coordinate information 103 on each point of interestincluded in the region management information 108.

In the case in which the coordinate information 103 on each point ofinterest is included in the additional information 106, the embeddingregion allocating means 23 allocates the embedding region for theadditional information 106 in the digitized output image 121 on thebasis of the image size information 102 from the image display means 22and the amount of additional information 107 on each point of interestincluded in the region management information 108 detected by theinformation detection means 21, and outputs the region information 109indicating the embedding region for the additional information 106.

An algorithm of allocating the embedding region for the additionalinformation 106 on the basis of the amount 107 of additional informationon each point of interest which the embedding region allocating means 23uses is predetermined between the embedding side and the detection sideso that it is the same as an algorithm of allocating the embeddingregion for the additional information 106 which the embedding regionallocating means 16 of the information processing apparatus 10 which isthe embedding side uses.

The information detection means 21 detects the additional information106, which is embedded in the watermarked image 110 by means of theelectronic watermarking method, from the watermarked image 110 on thebasis of the region information 109 outputted from the embedding regionallocating means 23, and outputs the detected additional information106.

The additional information reverse conversion means 24 carries outreverse conversion of the additional information 106 outputted from theinformation detection means 21, and outputs the text 105 of the relatedinformation and also outputs the mark information 104 on each point ofinterest in a case in which a mark indicating each point of interest isspecified.

In a case in which the coordinate information 103 on each point ofinterest is included in the additional information 106, the additionalinformation reverse conversion means 24 carries out reverse conversionof the additional information 106 outputted from the informationdetection means 21, and outputs the text 105 of the related informationand also outputs the mark information 104 on each point of interest whenthe mark indicating each point of interest is specified.

In the case in which a mark indicating each point of interest isspecified, the point of interest display means 25 displays the markbased on the mark information 104 outputted from the additionalinformation reverse conversion means 24 at a position specified by thecoordinate information 103 on each point of interest outputted from theembedding region allocating means 23 or the additional informationreverse conversion means 24 on the digitized output image. In contrast,in a case in which no mark indicating each point of interest isspecified, the point of interest display means 25 displays apredetermined mark at the specified position on the digitized outputimage 121.

The related information display means 26 accepts the text 105 of therelated information associated with a point of interest to be displayedfrom the additional information reverse conversion means 24, andcalculates and outputs the number 122 of characters in the text 105. Thedisplay position determining means 27 determines an optimal position ofdisplay of the related information so that the point of interest to bedisplayed is not hided by the corresponding related information on thebasis of the coordinate information 103 on the point of interestoutputted from the embedding region allocating means 23 or additionalinformation reverse conversion means 24, the mark information 104 fromthe additional information reverse conversion means 24, and the numberof characters 122 in the text 105 of the related information from therelated information display means 26, and outputs information 123 on thedetermined optimal display position.

The related information display means 26 displays the relatedinformation on the digitized output image 121 displayed by the imagedisplay means 22 on the basis of the information 123 on the determinedoptimal display position which is outputted from the display positiondetermining means 27. The point of interest display switching means 28makes the embedding region allocating means 23, point of interestdisplay means 25, and related information display means 26 carry theirrespective processes for each point of interest according to aninstruction from the user.

The display position information 123 determined by the display positiondetermining means 27 is required for the related information displaymeans 26 to determine a display area needed for the display of therelated information from the number 122 of characters in the text 105 ofthe related information on the point of interest to be displayed, and tospecify the display area. In a case in which the display area is arectangle, the coordinates of two opposite vertices can be defined asthe display position information 123. As an alternative, in a case inwhich the size of the display area is obvious for the relatedinformation display means 26, the coordinates of a predetermined vertexof a rectangle can be defined as the display position information 123.

In FIGS. 1 and 2, the mark information 104 on each point of interest canbe defined by graphic information including the shape and size of a marksuch as a point, the width of a line, and a pattern like a dashed lineif needed. In a case in which the point of interest specifying means 12of the information processing apparatus 10 which is the embedding sideshown in FIG. 1 does not specify the mark information 104 in advance,the point of interest display means 25 of the information processingapparatus 20 which is the detection side shown in FIG. 2 determines thedescription of a predetermined mark definition, and displays the pointof interest.

As explained above, the coordinate information 103 on the point ofinterest is included in either the region management information 108 orthe additional information 106. In FIG. 1, whether the coordinateinformation 103 on the point of interest which the point of interestspecifying means 12 outputs is inputted to either the embedding regionallocating means 16 or the additional information conversion means 15depends on which the region management information 108 or the additionalinformation 106 includes the coordinate information 103 on the point ofinterest. Similarly, in FIG. 2, whether the coordinate information 103on the point of interest is outputted to either the embedding regionallocating means 23 or the additional information reverse conversionmeans 24 depends on which the region management information 108 or theadditional information 106 includes the coordinate information 103 onthe point of interest in FIG. 1.

FIG. 3 is a diagram showing examples of the formats of the regionmanagement information 108 and additional information 106 in a case inwhich the coordinate information 103 on each point of interest isincluded in the region management information 108. FIGS. 3( a) and 3(b)show examples of the format of the region management information 108,and FIGS. 3( c) and 3(d) show examples of the format of the additionalinformation 106.

In FIG. 3( a), an amount 131 of region management information indicatingthe number of point of interests, coordinate information 103 a on afirst point of interest, an amount 107 a of additional information onthe first point of interest, coordinate information 103 b on a secondpoint of interest, and an amount 107 b of additional information on thesecond point of interest are included in the region managementinformation 108. This amount 131 of region management information isinformation for notifying the number of point of interests to theembedding region allocating means 23 of the information processingapparatus which is the detection side, and the embedding regionallocating means 23 allocates an embedding region for the additionalinformation 106 only a number of times corresponding to the number ofpoint of interests. In a case in which the format of each of the twopieces of coordinate information 103 a and 103 b on the first and secondpoint of interests is the same as the format of each of the amounts 107a and 107 b of additional information on the first and second point ofinterests, the amount 131 of region management information can be thenumber of bytes of the total size of the region management information108. In this case, by subtracting the number of bytes of the amount 131of region management information, which is a fixed size, from the totalnumber of bytes, and dividing the subtraction result by a number ofbytes which is equal to the sum of the amount of one set of point ofinterest coordinates 103, which is a fixed size, and the amount 107 ofadditional information, the number of point of interests can beacquired.

In FIG. 3( b), an end identifier 132 indicating an end at the end of theformat is included instead of the amount 131 of region managementinformation of FIG. 3( a), the embedding region allocating means 23allocates an embedding region for the additional information 106 untilit detects this end identifier 132.

In FIG. 3( c), a text 105 a of related information on the first point ofinterest and a text 105 b of related information on the second point ofinterest are included in the additional information 106. In FIG. 3( d),mark information 104 a on the first point of interest, the text 105 a ofthe related information on the first point of interest, mark information104 b on the second point of interest, and the text 105 b of the relatedinformation on the second point of interest are included in theadditional information 106.

FIG. 4 is a diagram showing examples of the formats of the regionmanagement information 108 and additional information 106 in a case inwhich the coordinate information 103 on each point of interest isincluded in the additional information 106. FIGS. 4( a) and 4(b) showexamples of the format of the region management information 108, andFIGS. 4( c) and 4(d) show examples of the format of the additionalinformation 106.

In FIG. 4( a), the amount 131 of region management informationindicating the number of point of interests, the amount 107 a ofadditional information on the first point of interest, and the amount107 b of additional information on the second point of interest areincluded in the region management information 108. In FIG. 4( b), an endidentifier 132 indicating an end at the end of the format is includedinstead of the amount 131 of region management information of FIG. 4(a).

In FIG. 4( c), the coordinate information 103 a on the first point ofinterest, text 105 a of the related information on the first point ofinterest, coordinate information 103 b on the second point of interest,and text 105 b of the related information on the second point ofinterest are included in the additional information 106. In FIG. 4( d),the coordinate information 103 a on the first point of interest, markinformation 104 a on the first point of interest, text 105 a of therelated information on the first point of interest, coordinateinformation 103 b on the second point of interest, mark information 104b on the second point of interest, and text 105 b of the relatedinformation on the second point of interest are included in theadditional information 106.

In the formats of the region management information 108 and additionalinformation 106 which are shown in FIGS. 3 and 4, the order of thecoordinate information 103 on each point of interest and the amount 107of additional information on each point of interest in the regionmanagement information 108, and the order of the coordinate information103 on each point of interest, mark information 104 on each point ofinterest, text 105 of the related information on each point of interest,and so on in the additional information 106 are not limited to theexamples shown in the figures. In either of the formats, the data fieldscan be interchanged.

FIG. 5 is a conceptual diagram for explaining an example of processingcarried out by the information processing apparatus 10 which is theembedding side. This example will be explained assuming that, as shownin FIG. 5( a), the original digital input image 101 is an image in whichtwo persons are captured as objects to be shot. As shown in FIG. 5( b),a first point of interest 141 a for a first object is specifiedaccording to the user's input from the point of interest specifyingmeans 12, and the related information associated with the first point ofinterest 141 a is inputted through the user's input from the relatedinformation input means 13. At that time, the mark information 104including the shape and size of a mark indicating the first point ofinterest 141 a can be simultaneously inputted through the point ofinterest specifying means 102. The embedding region allocating means 16then allocates an embedding region 142 a for the additional information106 and outputs it as region information 109 on the basis of the imagesize information 102 from the image display means 11 and the amount 107of additional information on the point of interest 141 a from theadditional information conversion means 15, and the informationembedding means 17 outputs the additional information 106 as regioninformation 109, and embeds it into the embedding region 142 a.

Next, as shown in FIG. 5( c), a second point of interest 141 b, which isa second object, and the related information on the second point ofinterest are specified according to the same procedure, the markinformation 104 on the second point of interest is inputted, anembedding region 142 b for the additional information 106 on the secondpoint of interest 141 b is allocated and the additional information 106is embedded in the embedding region 142 b so that the embedding region142 b for the additional information 106 on the second point of interest141 b does not overlap the embedding region 142 a for the additionalinformation 106 on the first point of interest 141 a. Finally, as shownin FIG. 5( d), the information embedding means 17 embeds the regionmanagement information 108 in an embedding region 143 which is apredetermined region. A starting point of the embedding region 143 canbe fixedly placed at the upper left corner of the original digital inputimage 101, as shown in this figure, or can be alternatively placed atanother position. This example of FIG. 5 is shown as an example in whichthe embedding region 143 in which the region management information isfinally embedded can overlap the region 141 a.

FIG. 6 is a block diagram showing the internal structure of theinformation embedding means 17. This information embedding means 17 isprovided with an additional information embedding means 31 and a regionmanagement information embedding means 32, and is so constructed as toembed the region management information 108 in the correspondingembedding region after embedding the additional information 106 on eachpoint of interest in the corresponding embedding region. The informationembedding means 17 shown in FIG. 6 can thus make the embedding region143 in which the region management information 108 is embedded overlapthe embedding region 142 a in which the additional information 106 isembedded, as shown in FIG. 5. In FIG. 6, the region information 109inputted to the region management information embedding means 32 isoutputted from the embedding region allocating means 16 in a case inwhich the region management information 108 is not embedded in thepredetermined region.

FIG. 7 is a conceptual diagram for explaining another example of theprocessing carried out by the information processing apparatus 10 whichis the embedding side. This example differs from that of FIG. 5 in thatafter the first and second point of interests 141 a and 141 b arespecified and the two pieces of related information on them areinputted, the embedding region 143 for the region management information108, the embedding region 142 a for the additional information 106 onthe first point of interest 141 a, and the embedding region 142 b forthe additional information 106 on the second point of interest 141 b areallocated so that they dot not overlap one another, and the regionmanagement information, and the two pieces of additional information onthe first and second point of interests are embedded into thecorresponding embedding regions, respectively. If the embedding region143 can be specified, as in the case of FIG. 5, it is not necessary tonecessarily place the embedding region at the upper left corner of theoriginal digital input image 101, unlike the case as shown in FIG. 7,and it is not necessary to place the embedding region 143 prior to theplacement of the embedding regions 142 a and 142 b.

FIG. 8 is a block diagram showing the internal structure of theinformation embedding means 17. This information embedding means 17 isprovided with a region management information embedding means 32 and anadditional information embedding means 31, and is so constructed as toembed the additional information 106 on each point of interest in thecorresponding embedding region after embedding the region managementinformation 108 into the corresponding embedding region. Because theembedding region 143 and embedding regions 142 a and 142 b are allocatedso that they do not overlap one another in the example shown in FIG. 7,the information embedding means 17 can have an internal structure whichis shown in FIG. 6 or 8 in detail. In FIG. 8, the region information 109inputted to the region management information embedding means 32 isoutputted from the embedding region allocating means 16 in the case inwhich the region management information 108 is not embedded in thepredetermined region.

In any of the examples shown in FIGS. 5 and 7, in the case in which theregion management information 108 is not embedded in the predeterminedregion, and the region information 109 indicating the embedding region143 for the region management information 108 outputted by the embeddingregion allocating means 16 is encrypted, the information processingapparatus can make the user perform an input of a keyword, an ID, or thelike to simply hide and incorporate the region information 109 into theinput value. Furthermore, the information processing apparatus can usethe input value as a cryptographic key so that the embedding region 143for the region management information 108 can be kept from beingcorrectly known without the use of the cryptographic key, and theembedding regions 142 a and 142 b for the additional information 106 onthe two point of interests which can be detected later cannot beacquired correctly. In this case, the detection and separation of theimage, region management information 108, and additional information 106can be made to become difficult. After the key is determined, the regioninformation 109 can be acquired and embedded in the correspondingembedding region. As an alternative, the key can be determined from theregion information 109 which is previously embedded in the correspondingembedding region.

FIG. 9 is a conceptual diagram explaining an example of processingcarried out by the information processing apparatus 20 which is thedetection side. In this example, it is assumed that there arerestrictions that the display screen of the image display means 22 ofthe information processing apparatus 20 which is the detection side issmall, and related information is displayed so that it is superimposedon two persons who are objects to be shot. The information detectionmeans 21 extracts the region management information 108 from thepredetermined region of the watermarked image 110, and the embeddingregion allocating means 23 determines the region information 109 fromthe amount 107 of additional information included in the regionmanagement information 108, and, in a case in which the coordinateinformation 103 on each point of interest is included in the regionmanagement information 108, also outputs the coordinate information 103on each point of interest.

As shown in FIG. 9( a), the information processing apparatus 20 displaysthe digitized output image 121 outputted from the information detectionmeans 21 on the image display means 22. When the user specifies a pointof interest through the point of interest display switching means 28,the embedding region allocating means 23 outputs the region information109 on the specified point of interest to the information detectionmeans 21. The information detection means 21 extracts the additionalinformation 106 from the watermarked image 110 using the regioninformation 109, and outputs the additional information 106. Theadditional information reverse conversion means 24 carries out reverseconversion of the additional information 106, and, in a case in whichthe text 105 of the related information and mark information 104 areincluded in the additional information 106, outputs the mark information104, or, in a case in which the coordinate information 103 on the pointof interest is included in the additional information 106, outputs thecoordinate information 103 on the point of interest.

In a case in which the coordinate information 103 on the point ofinterest from either the embedding region allocating means 23 or theadditional information reverse conversion means 24 and mark information104 are included in the additional information 106, the point ofinterest display means 25 displays the first point of interest 141 a onthe digitized output image 121 currently displayed on the image displaymeans 22, as shown in the FIG. 9( b), on the basis of the markinformation 104 from the additional information reverse conversion means24. At that time, the first point of interest 141 a is represented by apoint at the position specified by the coordinate information 103, or isrepresented by a mark based on the mark information 104, for example, acircle having a center at the position specified by, for example, thecoordinate information 103 on the point of interest. The relatedinformation display means 26 displays the text 105 of the relatedinformation in the form of a description 144 a of the relatedinformation on the digitized output image 121 currently displayed on theimage display means 22 so that it does not overlap the position of thedisplay of the first point of interest 141 a, as shown in FIG. 9( b), onthe basis of the display position information 123 determined by thedisplay position determining means 27.

When the user specifies the next point of interest through the point ofinterest display switching means 28, the information processingapparatus 20 carries out similar processing, and displays the secondpoint of interest 141 b with a mark such as a point or a quadrangle, asshown in FIG. 9( c), and also displays the text 105 of the relatedinformation on the second point of interest 141 b in the form of adescription 144 b of the related information so that it does not overlapthe second point of interest 141 b.

In the following processing, every time when the user specifies a pointof interest, each component shown in FIG. 2 carries out its processing.As an alternative, each component shown in FIG. 2 can carry out itsprocessing which it has to do before displaying a specified point ofinterest 141 and a description 144 of related information on thedigitized output image 121 without waiting for an instruction from theuser, and, after receiving the instruction from the user, can displaythe point of interest 141 and the description 144 of relatedinformation.

When the user performs a forward switching by manipulating a specificbutton or the like, or when the user places a cursor in the vicinity ofa point of interest or in a mark indicating a point of interest andmanipulates a specific button, the point of interest display switchingmeans 28 selects the point of interest as the point of interest to bedisplayed. The specified point of interest 141 and description 144 ofrelated information do not necessarily need to be displayedsimultaneously, and they can be alternately displayed one by oneaccording to manipulation of a button by the user. As an alternative,after a point of interest 141 currently being displayed is selected, acorresponding description 144 of related information can be displayed.

Each point of interest 141 indicated by mark information can becompletely independent from the display position of a correspondingdescription 144 of related information. In other words, each point ofinterest 141 indicated by mark information does not necessarily dependupon the display position of a corresponding description 144 of relatedinformation. In a case in which a relation between each point ofinterest 141 and a corresponding description 144 of related informationis not easy to identify, the relation can be expressed by, for example,a line showing the correspondence between them, as shown in FIGS. 9( b)and 9(c).

FIG. 10 is a block diagram showing the internal structure of theinformation detection means 21 of the information processing apparatus20 which is the detection side. This information detection means 21 isprovided with a region management information extraction means 41 and anadditional information extracting means 42, and is so constructed thatthe region management information extraction means 41 extracts theregion management information 108 first and the additional informationextracting means 42 then extracts the additional information 106 withthe region information 109. In FIG. 10, the region information 109inputted to the region management information extraction means 41 isoutputted from the embedding region allocating means 23 in the case inwhich the region management information 108 is not embedded in thepredetermined region.

FIG. 11 is a block diagram showing the internal structure of theinformation detection means 21 of the information processing apparatus20 which is the detection side. This information detection means 21 isprovided with a region management information extraction means 41, anadditional information extracting means 42, and a region managementinformation release means 43. After the region management informationextraction means 41 extracts the region management information 108embedded in the watermarked image 110 and outputs it to the embeddingregion allocating means 23, the region management information releasemeans 43 releases the region management information 108 embedded in thewatermarked image 110.

As shown in FIG. 5, in a case in which the embedding region 143 for theregion management information 108 overlaps an embedding region 142 foradditional information 106, a reversible electronic watermark is appliedfor embedding in the embedding region 143 region (C), and the regionmanagement information release means 43 of FIG. 11 releases the regionmanagement information 108. As a result, the number of times ofapplication of an electronic watermark embedded into an identicalposition can be reduced, degradation in the quality of the displayedimage can be reduced, and restrictions on detection by means of theelectronic watermark technique applied to embedding of the nextadditional information 106 can be also reduced. This method is effectiveeven in a case in which the embedding region 143 for the regionmanagement information 108 does not overlap any embedding region 142 foradditional information 106.

FIG. 12 is a block diagram showing the internal structure of theinformation detection means 21 of the information processing apparatus20 which is the detection side. This information detection means 21 isprovided with a region management information extraction means 41, anadditional information extracting means 42, a region managementinformation release means 43, and an additional information releasemeans 44. After the additional information extracting means 42 extractseach additional information 106 embedded in the watermarked image 110and outputs it to the additional information reverse conversion means24, the additional information release means 44 releases each additionalinformation 106 embedded in the watermarked image 110.

A reversible electronic watermark can be also applied to the embeddingregion 142 for the additional information 106, and the additionalinformation release means 44 shown in FIG. 12 can release the additionalinformation 106. In this case, the degradation in the quality of thedisplayed image can be further reduced. As shown in FIG. 12, theinformation detection means 21 additionally includes the regionmanagement information release means 43 and additional informationrelease means 44. As an alternative, the information detection means 21can additionally include only the additional information release means44.

An electronic watermark is fundamentally accompanied by only imagequality degradation having such a degree that human beings cannot sensewith their eyes, except for a case in which a large degree ofdegradation is intentionally introduced into the watermarked image.Therefore, by applying a reversible electronic watermark method to boththe embedding region 143 for the region management information 108 andthe embedding region 142 for the additional information 106, and byproviding the region management information release means 43 andadditional information release means 44 with the information detectionmeans 21, as shown in FIG. 12, the digitized output image 121 can bedisplayed with quality which is just the same as that of the originaldigital input image 101. However, even in the case in which theinformation detection means 21 additionally includes the regionmanagement information release means 43 and additional informationrelease means 44, the information processing apparatus 20 cantemporarily display the digitized output image 121 in which theelectronic watermark is embedded on the image display means 22, juts asit is, until it can display the digitized output image 121 in which theelectronic watermark is released.

It is desirable that the information processing apparatus 20 which isthe detection side takes protective measures toward protection of acopyright of the image data independently when outputting the digitizedoutput image 121 in which the electronic watermark is released tooutside the information processing apparatus 20. It is similarlydesirable to take protective measures toward protection of a copyrightof digital sound data included in the image data by means of theelectronic watermark or the like.

As mentioned above, in accordance with this embodiment 1, the embeddingregion allocating means 16 of the information processing apparatus 10which is the embedding side and the embedding region allocating means 23of the information processing apparatus 20 which is the detection sidecan allocate an embedding region having an optimal size or can allocatean embedding region having a minimum degree of redundancy on the basisof an amount 107 of additional information by only enabling the user tospecify a point of interest without allowing the user to directlyparticipate in the allocation of the embedding region. Therefore, thepresent embodiment offers an advantage of being able to embed additionalinformation 106 into the whole image region efficiently, thereby beingable to show only an intended region of interest effectively.

Furthermore, in accordance with this embodiment 1, the embedding regionallocating means 16 of the information processing apparatus 10 which isthe embedding side and the embedding region allocating means 23 of theinformation processing apparatus 20 which is the detection side canshare the region management information 108 and the region information109 indicating the embedding location for the additional information 106between them by embedding the coordinate information 103 on each pointof interest and the amount 107 of additional information specifying thesize of a related region by means of the electronic watermarking methodwithout specifying the position or shape of the embedding regiondirectly by means of the electronic watermarking method, andtransmitting them between them, and can distribute the regioninformation 109 common to both of them by using the same algorithm.Therefore, the present embodiment offers another advantage of being ableto, in a case in which the algorithm of determining the regioninformation 109 is not known, prevent the region information 109 whichis actually used from being distributed even if the coordinateinformation 103 and the amount 107 of additional information on eachpoint of interest are decrypted, thereby ensuring the security of therelated information embedded in the image.

Furthermore, in accordance with this embodiment 1, the embedding regionfor the additional information 106 which the embedding region allocatingmeans 16 of the information processing apparatus 10 which is theembedding side and the embedding region allocating means 23 of theinformation processing apparatus 20 which is the detection sideallocates to each point of interest has a simple shape, such as arectangle or a circle. Therefore, the present embodiment offers anadvantage of being able to allocate the embedding region for each pointof interest independently upon the position of each point of interest,thereby improving the flexibility of the allocation of the embeddingregion to each point of interest.

In addition, in accordance with this embodiment 1, the point of interestdisplay means 25 of the information processing apparatus 20 which is thedetection side displays each point of interest using a mark. Therefore,the present embodiment offers a further advantage of being able to makeit possible for the user of the embedding side to certainly inform theuser on the detection side of an intended point of interestindependently upon the embedding region for the additional information106.

Furthermore, while conventionally, a region of interest and an embeddingregion for related information are united and therefore the distinctionbetween them becomes blurry in a state in which the region of interestcannot but be enlarged superfluously, the information processingapparatus 20 in accordance with this embodiment 1 which is the detectionside can certainly inform the user of an intended point of interest 141by separately displaying each point of interest 141 and a correspondingdescription 144 of related information, as shown in FIG. 9.

In addition, the display position determining means 27 of theinformation processing apparatus 20 in accordance with this embodiment 1which is the detection side can determine an optimal display position ofthe related information so that the related information does not hidethe vicinity of the corresponding point of interest by referring to thecoordinate information 103 on the corresponding point of interest andthe number of characters of the text 105 of the related information inadvance even under the constraint that the related information cannotbut be superimposed on the currently-displayed image.

Embodiment 2

In accordance with above-mentioned embodiment 1, no special control ofthe total of the amount 107 of additional information which is embeddedinto the image by means of an electronic watermarking method isperformed. In contrast, in accordance with this embodiment 2, the totalof the amount 107 of additional information is controlled.

FIG. 13 is a block diagram showing the structure of informationprocessing apparatus 10 in accordance with embodiment 2 of the presentinvention which is an embedding side. This information processingapparatus 10 which is the embedding side includes a related informationamount adjustment means 18 in addition to the components of theinformation processing apparatus 10, as shown in FIG. 1 ofabove-mentioned embodiment 1, which is the embedding side, and the othercomponents of the information processing apparatus 10 of this embodimentare the same as those shown in FIG. 1.

Next, the operation of the information processing apparatus 10 which isthe embedding side will be explained.

The related information amount adjustment means 18 grasps a total amountof additional information to be embedded in the image on the basis ofimage size information 102 outputted from an image display means 11first, accepts an amount 107 of additional information on each point ofinterest which an additional information conversion means 15 outputs,outputs an amount 111 of remaining additional information which can beadditionally embedded in the original digital input image 101 to arelated information input means 13, and outputs a determined amount 112of additional information to an embedding region allocating means 16.

The related information input means 13 determines the amount of relatedinformation inputted from the user so that it is equal to or less thanthe amount 111 of remaining additional information outputted from therelated information amount adjustment means 18 to generate and output atext 105 of related information. The embedding region allocating means16 accepts the determined amount 112 of additional information from therelated information amount adjustment means 18, and then carries out thesame processing as explained in above-mentioned embodiment 1. The otherprocessing is the same as that explained in above-mentioned embodiment1.

As can be seen from the above description, this embodiment 2 providesthe same advantages as offered by above-mentioned embodiment 1.Furthermore, even when the size of the original digital input image 101is not sufficiently large and therefore the total amount of additionalinformation 106 to be embedded is restricted, the related informationamount adjustment means 18 can make a request of the user to inputrelated information having an amount commensurate with the amount 111 ofremaining additional information by always outputting the amount 111 ofremaining additional information which can be additionally embedded inthe original digital input image 101 to the related information inputmeans 13 so as to control the whole amount 107 of additionalinformation.

Embodiment 3

In above-mentioned embodiment 1, the information processing apparatus 20which is the detection side determines the position of display ofrelated information from the watermarked image 110. In contrast, inaccordance with this embodiment 3, the information processing apparatus10 which is the embedding side determines the position of display of therelated information, and embeds it, as a part of additional information106, in the image.

FIG. 14 is a block diagram showing the structure of the informationprocessing apparatus 10 which is the embedding side in accordance withembodiment 3 of the present invention. This information processingapparatus 10 which is the embedding side includes a display positiondetermining means 19 in addition to the components of the informationprocessing apparatus 10, as shown in FIG. 1 of above-mentionedembodiment 1, which is the embedding side, and the other components ofthe information processing apparatus 10 of this embodiment are the sameas those shown in FIG. 1.

Next, the operation of the information processing apparatus 10 which isthe embedding side will be explained.

The related information input means 13 accepts related informationrelated to each point of interest, and outputs a text 105 of the relatedinformation and also outputs the number 122 of characters included inthe text 105. For each point of interest, the display positiondetermining means 19 determines an optimal position of display of therelated information which does not hide each point of interest on thebasis of coordinate information 103 on each point of interest outputtedfrom the point of interest specifying means 12, mark information 104 oneach point of interest if needed, and the number 122 of charactersincluded in the text 105 of the related information outputted from therelated information input means 13, and outputs, as one element ofadditional information 106, display position information 123 on thedetermined optimal position of display of the related information to theadditional information conversion means 15.

The display position determining means 19 shown in FIG. 14 can producethe display position information 123 on the basis of the same method asthat of the display position determining means 27 shown in FIG. 2 ofabove-mentioned embodiment 1. The display position information 123produced by the display position determining means 19 can include notonly the information on the optimal position which does not hide eachpoint of interest and corresponding mark information 104, but alsoinformation on fine adjustments permitted for the user and informationon an intention of the user, such as making the display of the relatedinformation not hide a part of another background image.

FIG. 15 is a block diagram showing the structure of the informationprocessing apparatus 20 which is the detection side in accordance withembodiment 3 of the present invention. This information processingapparatus 20 which is the detection side is the same as the informationprocessing apparatus 20 which is the detection side, as shown in FIG. 2of above-mentioned embodiment 1, with the exception that the informationprocessing apparatus 20 of this embodiment 3 does not include thedisplay position determining means 27, and the other components of theinformation processing apparatus 20 of this embodiment are the same asthose shown in FIG. 2.

Next, the operation of the information processing apparatus 20 which isthe detection side will be explained.

The additional information reverse conversion means 24 detects, as apart of the additional information 106 of each point of interestacquired from the information detection means 21, the display positioninformation 123, and then outputs it to the related information displaymeans 26. Processes performed by the other components are the same asthose of above-mentioned embodiment 1.

FIG. 16 is a diagram showing examples of the format of the additionalinformation 106 in a case in which the coordinate information 103 oneach point of interest is included in the region management information108. In FIG. 16( a), display position information 123 a on relatedinformation on a first point of interest, a text 105 a of the relatedinformation on the first point of interest, display position information123 b on related information on a second point of interest, and a text105 b of the related information on the second point of interest areincluded in the additional information 106. In FIG. 16( b), the displayposition information 123 a on the related information on the first pointof interest, mark information 104 a on the first point of interest, text105 a of the related information on the first point of interest, displayposition information 123 b on the related information on the secondpoint of interest, mark information 104 b on the second point ofinterest, and text 105 b of the related information on the second pointof interest are included in the additional information 106. The formatof the region management information 108 in these cases has the sameexamples as shown in FIGS. 3( a) and 3(b) of above-mentioned embodiment1.

FIG. 17 is a diagram showing examples of the format of the additionalinformation 106 in a case in which the coordinate information 103 oneach point of interest is included in the additional information 106. InFIG. 17( a), the coordinate information 103 a on the first point ofinterest, display position information 123 a on the related informationon the first point of interest, text 105 a of the related information onthe first point of interest, coordinate information 103 b on the secondpoint of interest, display position information 123 b on the relatedinformation on the second point of interest, and text 105 b of therelated information on the second point of interest are included in theadditional information 106.

In FIG. 17( b), the coordinate information 103 a on the first point ofinterest, display position information 123 a on the related informationon the first point of interest, mark information 104 a on the firstpoint of interest, text 105 a of the related information on the firstpoint of interest, coordinate information 103 b on the second point ofinterest, display position information 123 b on the related informationon the second point of interest, mark information 104 b on the secondpoint of interest, and text 105 b of the related information on thesecond point of interest are included in the additional information 106.The format of the region management information 108 in these cases hasthe same examples as shown in FIGS. 4( a) and 4(b) of above-mentionedembodiment 1.

In the format of the additional information 106 which are shown in FIGS.16 and 17, the order of the display position information 123 on therelated information on each point of interest, mark information 104 oneach point of interest, text of the related information on each point ofinterest, and so on are not limited to the examples shown in thefigures. In the format, the data fields can be interchanged.

FIG. 18 is a diagram showing examples of the format of the regionmanagement information 108 in a case in which only the text 105 of therelated information on each point of interest is included in theadditional information 106. In this case, the additional information 106has a format as shown in FIG. 3( c) of above-mentioned embodiment 1.FIG. 18( a) shows a format in which a plurality of data fields aresuccessively arranged for each point of interest, and FIG. 18( b) showsa format which data on a plurality of point of interest are successivelyarranged for each data field. The arrangement as shown in FIG. 18( b)can be applied to the examples of the format of the region managementinformation 108 as shown in FIGS. 3( a) and 3(b) of above-mentionedembodiment 1.

In this case, the additional information conversion means 15 of FIG. 14accepts only the text 105 of the related information, and converts itinto additional information 106. The coordinate information 103 and markinformation 104 on each point of interest other than the text 105 of therelated information are inputted to the embedding region allocatingmeans 16, and are added to the region management information 108. Theadditional information reverse conversion means 24 of FIG. 15 extractsand outputs only the text 105 of the related information from theadditional information 106, and the embedding region allocating means 23extracts and outputs the coordinate information 103 and mark information104 on each point of interest from the region management information108.

FIG. 19 is a diagram showing an example of the format of the additionalinformation 106 in which a plurality of data fields are successivelyarranged for each point of interest. In the example, the format of theadditional information 106 shown in FIG. 17( b) is the one in which aplurality of data fields are successively arranged for each point ofinterest, as in the case of FIG. 18( b).

The example of the format of the region management information 108 shownin FIG. 18 and the example of the format of the additional information106 shown in FIG. 19 can be also applied to the information processingapparatus shown in above-mentioned embodiments 1 and 2, and aninformation processing apparatus shown in below-mentioned embodiment 4.In the examples shown in FIGS. 18 and 19, the number of data fieldsshown in FIG. 3 and FIG. 4 of above-mentioned embodiment 1 is maximized.This concept can be also applied to the other examples.

FIG. 20 is a conceptual diagram for explaining an example of processingperformed by the information processing apparatus 10 which is theembedding side. This example will be explained assuming that theoriginal digital input image 101 is an image in which two persons arecaptured as objects to be shot, as shown in FIG. 5 of embodiment 1. Asshown in FIG. 5( b), a first point of interest 141 a for a first objectis specified according to the user's input from the point of interestspecifying means 12, and the related information associated with thefirst point of interest 141 a is inputted through the user's input fromthe related information input means 13. At that time, the markinformation 104 including the shape and size of a mark indicating thefirst point of interest 141 a can be simultaneously inputted through thepoint of interest specifying means 12.

The display position determining means 19 then determines an optimalposition 145 a of display of the related information which does not hidethe point of interest 141 a on the basis of coordinate information 103on the point of interest 141 a outputted from the point of interestspecifying means 12, mark information 104 on the point of interest ifneeded, and the number 122 of characters included in the text 105 of therelated information outputted from the related information input means13, and outputs, as one element of additional information 106, displayposition information 123 on the determined optimal position of displayof the related information to the additional information conversionmeans 15. The information embedding means 17 then embeds the additionalinformation 106 in an embedding region 142 a outputted as regioninformation 109, as in the case of FIG. 5( b) of above-mentionedembodiment 1.

Next, as shown in FIG. 20( c), a second point of interest 141 b, whichis a second object, and an optimal position 145 b of display of therelated information which does not hide the point of interest 141 b aredetermined according to the same procedure, mark information 104 on thesecond point of interest is inputted, an embedding region 142 b for theadditional information on the second point of interest is allocated andthe additional information 106 is embedded in the embedding region 142b. Finally, as shown in FIG. 20( d), the information embedding means 17embeds the region management information 108 in an embedding region 143,as in the case of FIG. 5( d) of above-mentioned embodiment 1.

FIG. 21 is a conceptual diagram for explaining another example of theprocessing performed by the information processing apparatus 10 which isthe embedding side. This example differs from that of FIG. 20 in that,as shown in FIG. 21( b), after the first and second point of interests141 a and 141 b are specified and at least the optimal positions 145 aand 145 b of display of the two pieces of related information aredetermined, the embedding region 143 for the region managementinformation 108, the embedding region 142 a for the additionalinformation 106 on the first point of interest 141 a, and the embeddingregion 142 b for the additional information 106 on the second point ofinterest 141 b are allocated so that they dot not overlap one another,and the region management information, and the two pieces of additionalinformation on the first and second point of interests are embedded intothe corresponding embedding regions, respectively, as shown in FIGS. 21(c) and 21(d). If the embedding region 143 can be specified, as in thecase of FIG. 20, it is not necessary to necessarily place the embeddingregion at the upper left corner of the original digital input image 101,unlike the case as shown in FIG. 21, and it is not necessary to placethe embedding region 143 prior to the placement of the embedding regions142 a and 142 b.

As can be seen from the above description, this embodiment 3 providesthe same advantages as offered by above-mentioned embodiment 1.Furthermore, because even under the constraint that the relatedinformation cannot but be superimposed on the currently-displayed image,the display position determining means 19 of the information processingapparatus 10 which is the embedding side can determine the position 145of display of the related information on each point of interest, thisembodiment 3 offers another advantage of being able to display therelated information at an optimal display position to which the user hasmade a fine adjustment to reflect his or her intention in the displayposition without having to provide the information processing apparatus20 which is the detection side with the display position determiningmeans 27.

Embodiment 4

An information processing apparatus 10 in accordance with thisembodiment 4 which is an embedding side performs both adjustment of theamount of additional information, as explained in above-mentionedembodiment 2, and determination of the display position of relatedinformation, as explained in above-mentioned embodiment 3.

FIG. 22 is a block diagram showing the structure of informationprocessing apparatus 10 in accordance with embodiment 4 of the presentinvention which is the embedding side. This information processingapparatus 10 which is the embedding side includes a related informationamount adjustment means 18 in addition to the components of theinformation processing apparatus 10 which is the embedding side as shownin FIG. 14 of above-mentioned embodiment 3, and the other components ofthe information processing apparatus 10 of this embodiment are the sameas those shown in FIG. 14.

Processing performed by the related information amount adjustment means18 of the information processing apparatus 10 which is the embeddingside is the same as that of above-mentioned embodiment 2, processingperformed by a display position determining means 19 is the same as thatof above-mentioned embodiment 3, and processes performed by the othercomponents are the same as those of above-mentioned embodiment 1.

As mentioned above, in accordance with this embodiment 4, the sameadvantages as provided by above-mentioned embodiment 1 are offered, andthe same advantages as provided by above-mentioned embodiments 2 and 3can be simultaneously offered.

The information processing apparatus shown in each of above-mentionedembodiments 1 to 4 can be implemented via hardware. As an alternative,the information processing apparatus shown in each of above-mentionedembodiments 1 to 4 can be implemented via a software program which runson a general-purpose apparatus, such as a personal computer (referred toas a PC from here on). The information processing apparatus shown ineach of above-mentioned embodiments 1 to 4 can be alternativelyimplemented via a combination of hardware and software.

In the information processing apparatus shown in each of above-mentionedembodiments 1 to 4, the embedding region for each of the regionmanagement information 108 and the additional information 106 on eachpoint of interest is a single region. However, it is not necessary tolimit the embedding region to a single region and the shape of theembedding region is not limited to a rectangle and can be freely definedas long as the embedding region allocating means 16 and 23 can divideeach embedding region into two or more subregions and carry outintegrated management of the two or more subregions on the basis of theimage size information 102 and the amount 107 of additional information.

In the information processing apparatus shown in each of above-mentionedembodiments 1 to 4, the electronic watermarking method which theinformation embedding means 17 and information detection means 21 use isnot limited to the method disclosed in above-mentioned patent reference1, and another method can be used as the electronic watermarking method.

Embodiment 5

In this embodiment 5, a digital camera in which the informationprocessing apparatus 10 which is the embedding side, as explained ineither of above-mentioned embodiments 1 to 4, is mounted will beexplained.

FIG. 23 is a block diagram showing the structure of the digital camera210 in accordance with embodiment 5 in which the information processingapparatus 10 in accordance with the present invention which is theembedding side is mounted. This digital camera 210 is provided with animaging means 211, the information processing apparatus 10 which is theembedding side, and a storage means 212.

Next, the operation of the digital camera in accordance with thisembodiment of the present invention will be explained.

The imaging means 211 shoots an object to be shot to produce an image,and outputs it as an original digital input image 101. As explained inabove-mentioned embodiments 1 to 4, the information processing apparatus10 which is the embedding side embeds region management information 108and additional information 106 in the original digital input image 101by means of an electronic watermarking method, and outputs a watermarkedimage 110. The storage means 212 stores the watermarked image 110therein, and outputs it to outside itself via a medium/external device213.

As an alternative, before the electronic watermark is embedded in theoriginal digital input image 101, the original digital input image 101can be read out again after being stored in the storage means 212, andthe information processing apparatus 10 which is the embedding side canembed the region management information 108 and additional information106 in the original digital input image by means of the electronicwatermarking method. Furthermore, the digital camera read the originaldigital input image 101 from the medium/external device 213 and write itin the storage means 212, and apply embedding using the electronicwatermarking method to the original digital input image 101.

The medium in the medium/external device 213 is referred to as asemiconductor memory medium, a disk medium, or a card medium in whichdata can be recorded electrically, magnetically, or optically, andrepresents a mode in which data is written in or read out of the mediumin order to exchange image data inside the storage means 212. The mediumdoes not depend on characteristics such as read-only and rewritable. Theexternal device in the medium/external device 213 represents a mode inwhich data is outputted and written in a medium or the like by cable orradio, or data is inputted and read out in order to exchange image dataoutside the storage means 212. The external device does not depend on aconnection interface with the medium/external device 213.

As mentioned above, this embodiment 5 offers an advantage of being ableto apply the information processing apparatus 10 in accordance witheither of above-mentioned embodiments 1 to 4 which is the embedding sideto the digital camera 210.

Embodiment 6

In this embodiment 6, a camera-equipped mobile phone or a PC in whichthe information processing apparatus 10 which is the embedding side andthe information processing apparatus 20 which is the detection side, asexplained in either of above-mentioned embodiments 1 to 4, are mountedwill be explained.

FIG. 24 is a block diagram showing the structure of the camera-equippedmobile phone 220 or PC 230 in accordance with embodiment 6 in which themounted information processing apparatus 10 which is the embedding sidein accordance with the present invention is mounted. Thiscamera-equipped mobile phone 220 or PC 230 is provided with an imagingmeans 221, the information processing apparatus 10 which is theembedding side, a storage means 222, a transmitting means 223, and areceiving means 224.

Next, the operation of the camera-equipped mobile phone 220 or PC 230shown in FIG. 24 will be explained.

The imaging means 221 shoots an object to be shot to produce an image,and outputs it as an original digital input image 101. As explained ineither of above-mentioned embodiments 1 to 4, the information processingapparatus 10 which is the embedding side embeds region managementinformation 108 and additional information 106 in the original digitalinput image 101 by means of an electronic watermarking method, andoutputs a watermarked image 110. The transmitting means 223 sends outthe watermarked image 110 to outside the camera-equipped mobile phone orPC through communications 226.

Before the region management information and additional information areembedded in the original digital input image 101 by means of theelectronic watermarking method, the original digital input image 101 canbe read out again after being stored in the storage means 222, and theregion management information and additional information can be embeddedin the original digital input image by means of the electronicwatermarking method. As an alternative, the original digital input image101 is retrieved from outside the camera-equipped mobile phone or PC bythe receiving means 224 through the communications 226 and is thenstored in the storage means 222, and, after that, the region managementinformation and additional information can be embedded in the originaldigital input image by means of the electronic watermarking method. Theoriginal digital input image retrieved from outside the camera-equippedmobile phone or PC by the receiving means 224 can be alternativelysubjected to the embedding of the region management information andadditional information therein by means of the electronic watermarkingmethod without being stored in the storage means. As an alternative, theoriginal digital input image 101 can be inputted directly to thecamera-equipped mobile phone or PC from a medium/external device 225 viathe storage means 222.

The watermarked image 110 can be alternatively stored in the storagemeans 222 before being transmitted to outside the camera-equipped mobilephone or PC, and can be read again and transmitted to outside thecamera-equipped mobile phone or PC. As an alternative, the watermarkedimage 110 can be transmitted directly to outside the camera-equippedmobile phone or PC from the medium/external device 225.

The communications 226 represent communications by cable or by radio,and do not depend upon the modulating method, amplitudes, wavelengths,difference between analog and digital, etc. Furthermore, thecommunications 226 do not depend upon the type of a network via whichthe communications are carried out, such as a private network or widearea network, like the Internet, a temporary connection or constantconnection network, or a dedicated network or public network.

FIG. 25 is a block diagram showing the structure of the camera-equippedmobile phone 240 or PC 250 in accordance with embodiment 6 in which theinformation processing apparatus 20 which is the detection side inaccordance with the present invention is mounted. This camera-equippedmobile phone 240 or PC 250 is provided with a receiving means 241, theinformation processing apparatus 20 which is the detection side, astorage means 242, and a transmitting means 243.

Next, the operation of the camera-equipped mobile phone 240 or PC 250shown in FIG. 25 will be explained.

The receiving means 241 retrieves a watermarked image 110 from outsidethe camera-equipped mobile phone or PC through communications 245. Asexplained in above-mentioned embodiments 1 to 4, the informationprocessing apparatus 20 which is the detection side detects regionmanagement information 108 and additional information 106 from thewatermarked image 110 by means of an electronic watermarking method, andoutputs a digitized output image 121. The storage means 242 stores thedigitized output image 121 therein, and outputs it to outside thecamera-equipped mobile phone or PC via a medium/external device 244.

As an alternative, the watermarked image 110 can be stored in thestorage means 242 before the region management information 108 andadditional information 106 are detected from the watermarked image bymeans of the electronic watermarking method, and, after that, thewatermarked image 110 can be read again and the region managementinformation and additional information can be detected from thewatermarked image by means of the electronic watermarking method.Alternatively, after the watermarked image 110 is inputted directly tothe camera-equipped mobile phone or PC from the medium/external device244 via the storage means 242, the region management information 108 andadditional information 106 can be detected from the watermarked image bymeans of the electronic watermarking method.

Furthermore, the digitized output image 121 can be alternativelytransmitted to outside the camera-equipped mobile phone or PC by thetransmitting means 243, just as it is, through the communications 245.As an alternative, after the digitized output image is stored in thestorage means 242 before being transmitted, the digitized output imagecan be read again and transmitted to outside the camera-equipped mobilephone or PC.

As mentioned above, this embodiment 6 offers an advantage of being ableto apply the information processing apparatus 10 which is the embeddingside and information processing apparatus 20 which is the detection sidein accordance with either of above-mentioned embodiments 1 to 4 to thecamera-equipped mobile phone 220 or PC 230, and the camera-equippedmobile phone 240 or PC 250, respectively.

Embodiment 7

In this embodiment 7, a printer in which the information processingapparatus 20 which is the detection side as explained in either ofabove-mentioned embodiments 1 to 4 is mounted will be explained.

FIG. 26 is a block diagram showing the structure of the printer 260 inaccordance with embodiment 7 in which the information processingapparatus 20 which is the detection side in accordance with the presentinvention is mounted. This printer 260 is provided with the informationprocessing apparatus 20 which is the detection side, a storage means261, and a recording means 262.

Next, the operation of the printer in accordance with this embodiment ofthe present invention will be explained.

The information processing apparatus 20 which is the detection sidedetects region management information 108 and additional information 106from a watermarked image 110 inputted thereto from outside the printervia a medium/external device or PC 263 by means of an electronicwatermarking method, and outputs a digitized output image 121, asexplained in above-mentioned embodiments 1 to 4. The recording means 262outputs the digitized output image 121 as shown in FIG. 9 afterincorporating a display of related information on a point of interestinto the digitized output image.

As an alternative, the watermarked image 110 can be stored in thestorage means 261 before the region management information 108 andadditional information 106 are detected from the watermarked image bymeans of the electronic watermarking method, and, after that, thewatermarked image 110 can be read again and the region managementinformation and additional information can be detected from thewatermarked image by means of the electronic watermarking method.Furthermore, the digitized output image 121 can be stored in the storagemeans 261 before related information is recorded therein by therecording means 262, and, after that, can be read again and then relatedinformation can be recorded in the read digitized output image.

As mentioned above, this embodiment 7 offers an advantage of being ableto apply the information processing apparatus 20 which is the detectionside in accordance with either of above-mentioned embodiments 1 to 4 tothe printer 260.

The digital camera 210, camera-equipped mobile phone 220 or 240 or PC230 or 250, and printer 260 are examples of the application of theinformation processing apparatus shown in above-mentioned embodiments 5to 7. The information processing apparatus shown in above-mentionedembodiments 5 to 7 can be also applied to other applications having thesame structures as the digital camera, camera-equipped mobile phone orPC, and printer, respectively. For example, in the structure shown inFIG. 26 of above-mentioned embodiment 7, in a case in which a scanner isconnected, as an imaging means, to the printer via the medium/externaldevice or PC 263, a facsimile apparatus can be implemented with the samestructure.

In each of those applications mentioned above, not only the functions ofthe information processing apparatus in accordance with the presentinvention can be incorporated, as hardware, into an existing device, butalso software (plug-in or add-in software) can be read and incorporatedinto the existing device via a medium or communications. In thisvariant, the functionality of the existing device can be extendedadditionally and the software version can be upgraded to the latestversion. Furthermore, the existing device itself whose functionality canbe extended through the reading of software can be software.

INDUSTRIAL APPLICABILITY

As mentioned above, the information processing apparatus and theinformation processing method in accordance with the present inventionrefers to an amount of additional information and allocates an embeddingregion when embedding additional information on each point of interestin an image by means of an electronic watermarking method, and aresuitable for use in a digital camera, a camera-equipped mobile phone, aPC, a printer, and so on.

1. An information processing apparatus comprising: an image displaymeans for displaying an original digital input image, and for outputtinginformation on a size of the image; a point of interest specifying meansfor specifying a point of interest in the original digital input imagecurrently displayed on a basis of an inputted instruction; a relatedinformation output means for inputting related information on said pointof interest, and for outputting a text of the related information; anadditional information conversion means for converting the text of saidrelated information to additional information and outputting thisadditional information, and for outputting an amount of the additionalinformation on said each point of interest; an embedding regionallocating means for outputting region management information includingsaid amount of additional information, for allocating an embeddingregion for said additional information in said original digital inputimage on a basis of said image size information and said amount of theadditional information, and for outputting information on said embeddingregion as region information; and an embedding information means forembedding said region management information in a predetermined regionin said original digital input image, and also embedding said additionalinformation in said original digital input image on a basis of saidregion information to generate and output an watermarked image.
 2. Theinformation processing apparatus according to claim 1, characterized inthat said point of interest specifying means outputs coordinateinformation on the specified point of interest, and said embeddingregion allocating means outputs said region management information afterincorporating said coordinate information on the specified point ofinterest into said region management information.
 3. The informationprocessing apparatus according to claim 1, characterized in that saidpoint of interest specifying means outputs coordinate information on thespecified point of interest, and said additional information conversionmeans outputs said additional information after incorporating saidcoordinate information on the specified point of interest into saidadditional information.
 4. The information processing apparatusaccording to claim 1, characterized in that said point of interestspecifying means outputs mark information on the specified point ofinterest, and said additional information conversion means outputs saidadditional information after incorporating said mark information on thespecified point of interest into said additional information.
 5. Theinformation processing apparatus according to claim 1, characterized inthat said information processing apparatus includes a relatedinformation adjustment means for outputting an amount of remainingadditional information which can be additionally embedded on a basis ofsaid image size information and said amount of additional information,and said related information input means inputs the related informationon a basis of said amount of remaining additional information.
 6. Theinformation processing apparatus according to claim 1, characterized inthat said point of interest specifying means outputs coordinateinformation on the specified point of interest, said related informationinput means outputs a number of characters included in the text of theinputted related information, and characterized in that said informationprocessing apparatus includes a display position determining means fordetermining a display position at which said related information is tobe displayed in said original digital input image so that said relatedinformation does not overlap the point of interest to be displayed on abasis of said coordinate information on the specified point of interestand said number of characters included in the text of the inputtedrelated information, and said additional information conversion meansoutputs said additional information after incorporating said displayposition information into said additional information.
 7. Theinformation processing apparatus according to claim 6, characterized inthat said point of interest specifying means outputs mark information onthe specified point of interest, and said display position determiningmeans determines said display position in consideration of said markinformation on the specified point of interest.
 8. An informationprocessing apparatus comprising: an information detection means fordetecting and outputting region management information from apredetermined region of a watermarked image in which additionalinformation including a text of related information on a point ofinterest and region management information including an amount ofadditional information for said each point of interest are embedded, fordetecting and outputting said additional information from saidwatermarked image on a basis of region information mentioned below, andfor outputting a digitized output image from said watermarked image; animage display means for displaying said digitized output image, and foroutputting image size information on a size of said digitized outputimage; an embedding region allocating means for allocating an embeddingregion for said additional information in said digitized output image ona basis of said image size information and an amount of additionalinformation included in said region management information, and foroutputting information on said embedding region as region information;an additional information reverse conversion means for carrying outreverse conversion of the additional information detected by saidinformation detection means, and for outputting the text of the relatedinformation on the point of interest; a point of interest display meansfor displaying a point of interest which is specified in said digitizedoutput image on a basis of coordinate information on the specified pointof interest included in either said region management information orsaid additional information; a related information display means forinputting the text of the related information on said point of interest,for outputting a number of characters included in said text, and fordisplaying the related information in said digitized output image on abasis of display position information mentioned below; and a displayposition determining means for determining a display position at whichthe related information is to be displayed in the original digital inputimage so that said related information does not overlap the point ofinterest to be displayed on a basis of the coordinate information onsaid point of interest and said number of characters included in thetext of the related information.
 9. The information processing apparatusaccording to claim 8, characterized in that said point of interestdisplay means displays the point of interest on a basis of markinformation included in said additional information, and said displayposition determining means determines a position of display of therelated information in consideration of said mark information.
 10. Theinformation processing apparatus according to claim 8, characterized inthat said information detection means detects said region managementinformation, and releases said region management information from saidwatermarked image after outputting said region management information.11. The information processing apparatus according to claim 8,characterized in that said information detection means detects saidadditional information, and releases said additional information afteroutputting said watermarked image.
 12. An information processingapparatus comprising: an information detection means for detecting andoutputting region management information from a predetermined region ofa watermarked image in which additional information including a text ofrelated information on a point of interest and region managementinformation including an amount of additional information for said eachpoint of interest are embedded, for detecting and outputting saidadditional information from said watermarked image on a basis of regioninformation mentioned below, and for outputting a digitized output imagefrom said watermarked image; an image display means for displaying saiddigitized output image, and for outputting image size information on asize of said digitized output image; an embedding region allocatingmeans for allocating an embedding region for said additional informationin said digitized output image on a basis of said image size informationand an amount of additional information included in said regionmanagement information, and for outputting information on said embeddingregion as region information; an additional information reverseconversion means for carrying out reverse conversion of the additionalinformation detected by said information detection means, and foroutputting the text of the related information on a point of interestand display position information indicating a position of display ofsaid related information at which it does not overlap the point ofinterest to be displayed; a related information display means fordisplaying the specified point of interest in said digitized outputimage on a basis of coordinate information on the point of interestincluded in either said region management information or said additionalinformation; and a related information display means for inputting thetext of the related information on said point of interest, and fordisplaying the related information in said digitized output image on abasis of said display position information.
 13. The informationprocessing apparatus according to claim 12, characterized in that saidpoint of interest display means displays the point of interest on abasis of mark information included in said additional information. 14.An information processing method comprising the steps of: displaying anoriginal digital input image, and outputting information on a size ofthe image; specifying a point of interest in the original digital inputimage currently displayed on a basis of an inputted instruction;inputting related information on said point of interest, and outputtinga text of the related information; converting the text of said relatedinformation into additional information and outputting this additionalinformation, and outputting an amount of the additional information onsaid each point of interest; outputting region management informationincluding said amount of additional information, allocating an embeddingregion for said additional information in said original digital inputimage on a basis of said image size information and said amount of theadditional information, and outputting information on said embeddingregion as region information; and embedding said region managementinformation in a predetermined region in said original digital inputimage, and also embedding said additional information in said originaldigital input image on a basis of said region information to generateand output a watermarked image.
 15. An information processing methodcomprising the steps of: detecting and outputting region managementinformation from a predetermined region of a watermarked image in whichadditional information including a text of related information on apoint of interest and region management information including an amountof additional information for said each point of interest are embedded;outputting a digitized output image from said watermarked image;displaying said digitized output image, and outputting image sizeinformation on a size of said digitized output image; allocating anembedding region for said additional information in said digitizedoutput image on a basis of said image size information and an amount ofadditional information included in said region management information,and outputting information on said embedding region as regioninformation; detecting and outputting the additional information fromsaid watermarked image on a basis of said region information; carryingout reverse conversion of said detected additional information, andoutputting the text of the related information on the point of interest;displaying a point of interest which is specified in said digitizedoutput image on a basis of coordinate information on the point ofinterest included in either said region management information or saidadditional information; inputting the text of the related information onsaid point of interest, and outputting a number of characters includedin said text; determining a display position at which the relatedinformation is to be displayed in the original digital input image sothat said related information does not overlap the point of interest tobe displayed on a basis of the coordinate information on said point ofinterest and said number of characters included in the text of therelated information, and outputting the display position information;and inputting the text of said related information, and displaying therelated information in said digitized output image on a basis of saiddisplay position information.
 16. An information processing methodcomprising the steps of: detecting and outputting region managementinformation from a predetermined region of a watermarked image in whichadditional information including a text of related information on apoint of interest and region management information including an amountof additional information for said each point of interest are embedded;outputting a digitized output image from said watermarked image;displaying said digitized output image, and outputting image sizeinformation on a size of said digitized output image; allocating anembedding region for said additional information in said digitizedoutput image on a basis of said image size information and an amount ofadditional information included in said region management information,and outputting information on said embedding region as regioninformation; detecting and outputting the additional information fromsaid watermarked image on a basis of said region information; carryingout reverse conversion of said detected additional information, andoutputting the text of the related information on a point of interestand display position information indicating a position of display ofsaid related information at which it does not overlap the point ofinterest to be displayed; displaying the specified point of interest insaid digitized output image on a basis of coordinate information on thepoint of interest included in either said region management informationor said additional information; and inputting the text of said relatedinformation, and displaying the related information in said digitizedoutput image on a basis of said display position information.